Raising the Standards of AI Products

We propose a mechanism for the promotion of high-standards in commercial Artificial Intelligence products, namely an association of companies which would regulate their own membership using a code of practice and the precedents set by previous cases. Membership would provide some assurance of quality. We argue the benefits of such a mechanism, and discuss some of the details including the proposal of a code of practice. This paper is intended as a vehicle for discussion rather than as the presentation of a definitive solution. Acknowledgements We are grateful to members of the Edinburgh Computing and Social Responsibility Group for feedback on an earlier draft of the code, and to Maggie Boden for comments on the paper itself. 1. The Need for High Standards in AI Products Credibility has always been a precious asset for AI, but never more so than now. The current commercial interest in AI is giving us the chance to prove ourselves. If the range of AI products now coming onto the market are shown to provide genuine solutions to hard problems then we have a rosy future. A few such useful products have been produced, but our future could still be jeopardised by a few, well publicised, failures. Genuine failures where there was determined, but ultimately unsuccessful, effort to solve a problem are regrettable, but not fatal. Every technology has its limitations. What we have to worry about are charlatans and incompetents taking advantage of the current fashion and selling products which are overrated or useless. AI might then be stigmatised as a giant con-trick, and the current tide of enthusiasm would ebb as fast as it flowed. (Remember Machine Translation it could still happen.) Both companies selling AI products and academic AI research groups would suffer in the resulting crash. AI companies are very dependent on the good-will of their customers. The current life-span of typical AI products is about 1-5 years. The customers of AI products are likely to stay in the market for several times this period; typically they are themselves companies or academic groups engaged in AI research or interested in the long term application of AI techniques. To stay in business the AI company must sell successive upgrades of its products to the same group of customers and, therefore, must build up and maintain a good reputation. If AI business is to expand then new customers must be brought into this existing group. This will only happen if the overall range of AI products is of high-quality and the reputation of this particular company is good. Thus it is in the interests of each company to raise both the general and its particular standard. It must also convince customers that its products are of high standard. When the market was small a company with highquality products could win new customers by word of mouth. Now the market is growing they must use advertisements, and it becomes harder for a company to convince potential customers that its products are of high quality. Apart from improving the public image of AI and increasing the market for AI products, producing more high-quality products would raise morale and standards in AI itself, leading to a virtuous circle of standards being raised, better work being done, good people being attracted to the field, and even more high quality products emerging. Poor-quality products will produce a vicious circle going in the opposite direction. But these internal reasons for wanting high standards, while important to insiders, are perhaps less important than external reasons. AI products look destined to play a major role in society. That society deserves, and has the right to expect, protection from exploitation by AI companies and from being harmed by AT products. An extreme, potential example of such harm is described in [Thompson et al 84], which argues that it is not possible to build an automatic or semiautomatic launch-on-warning system for nuclear weapons with anything like an acceptable failure rate. Anybody who claimed to have done so, or who claimed to be able to do so, would be guilty of misleading the public in a way that could have disastrous consequences. If such a claimant were an AI company then the whistle might be blown on it by the mechanism described below. However, the main purpose of this proposal is to catch less apocalyptic, but more common-place, mis1290 A. Bundy and R. Clutterbuck leading claims, whether or not they might give rise to a legal remedy. Examples might be an expert systems shell whose advertised range far exceeds the problems it is really suitable for, or a natural language front end which is presented as being able to deal with a much wider input than it, in fact, can. 2. A Professional Association The academic field guards itself against charlatans and incompetents by the peer review of research papers, grants, PhDs, etc. There is no equivalent safeguard in the commercial AI field. Faced with this problem other fields set up professional associations and codes of practice. AI needs a similar set-up. We propose that the responsible AI companies should get together now to found such an association. Continued membership should depend on a constant high-standard of AI products and in-house expertise. Members would be able to advertise their membership, and customers would have some assurance of quality. Charlatans and incompetents would be excluded or ejected, so that the failure of their products would not be seen to reflect on the field as a whole nor on the companies in the Association. Since the trade in AI products is international, with multi-national companies involved both as vendors and customers, the Association would also need to be international. Otherwise, there would be difficulty over membership of multi-national companies and about dealing with complaints resulting from international sales. It is particularly important to make membership attractive to multi-nationals because, with their existing reputation, they have less to gain from the cachet of membership and are more able to avoid the full impact of national registration. The Association would be self-regulating. If its decisions were seen to be too arbitrary then the value of Association membership would be devalued in the eyes of the public, the customers and the vendors, and the importance of its decisions would decrease in proportion; customers would take no account of Association membership when deciding to buy, so vendors would not bother to join. For such self-regulation to work it is necessary for the Association to be publically visible. Both vendors and potential customers must be aware of the Association and must see its decisions as fair and its sanctions as effective. Customers will then use Association membership as a major determinant when deciding whether and what product to buy. Vendors will regard Association membership as a valuable asset to their company, and will aim for high quality in their products in order to retain membership. They will want to use Association membership in their advertising, and this will, in turn, improve the visibility of the Association. It will be necessary for the Association to maintain a high profile of both its existence and its actions, to be E.g. the Vehicle Builders and Repairers Association and the National Association of Estate Agents. Note that, unless otherwise stated, all examples are of UK institutions or laws. open about its decisions, and to employ effective